Adaptive robotic interface apparatus and methods

ABSTRACT

Apparatus and methods for training of robotic devices. A robot may be trained by a user guiding the robot along target trajectory using a control signal. A robot may comprise an adaptive controller. The controller may be configured to generate control commands based on the user guidance, sensory input and a performance measure. A user may interface to the robot via an adaptively configured remote controller. The remote controller may comprise a mobile device, configured by the user in accordance with phenotype and/or operational configuration of the robot. The remote controller may detect changes in the robot phenotype and/or operational configuration. User interface of the remote controller may be reconfigured based on the detected phenotype and/or operational changes.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is related to co-pending and co-owned U.S. patentapplication Ser. No. 13/842,530 entitled “ADAPTIVE PREDICTOR APPARATUSAND METHODS”, filed Mar. 15, 2013, U.S. patent application Ser. No.13/842,562 entitled “ADAPTIVE PREDICTOR APPARATUS AND METHODS FORROBOTIC CONTROL”, filed Mar. 15, 2013, U.S. patent application Ser. No.13/842,616 entitled “ROBOTIC APPARATUS AND METHODS FOR DEVELOPING AHIERARCHY OF MOTOR PRIMITIVES”, filed Mar. 15, 2013, U.S. patentapplication Ser. No. 13/842,647 entitled “MULTICHANNEL ROBOTICCONTROLLER APPARATUS AND METHODS”, filed Mar. 15, 2013, and U.S. patentapplication Ser. No. 13/842,583 entitled “APPARATUS AND METHODS FORTRAINING OF ROBOTIC DEVICES”, filed Mar. 15, 2013, each of the foregoingbeing incorporated herein by reference in its entirety.

COPYRIGHT

A portion of the disclosure of this patent document contains materialthat is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure, as it appears in the Patent and TrademarkOffice patent files or records, but otherwise reserves all copyrightrights whatsoever.

BACKGROUND

1. Technological Field

The present disclosure relates to adaptive control and training ofrobotic devices.

2. Background

Robotic devices are used in a variety of applications, such asmanufacturing, medical, safety, military, exploration, and/or otherapplications. Some existing robotic devices (e.g., manufacturingassembly and/or packaging) may be programmed in order to perform desiredfunctionality. Some robotic devices (e.g., surgical robots) may beremotely controlled by humans, while some robots (e.g., iRobot Roomba®)may learn to operate via exploration.

Robotic devices may comprise hardware components that may enable therobot to perform actions in 1, 2, and/or 3-dimensional space. Somerobotic devices may comprise one or more components configured tooperate in more than one spatial dimension (e.g., a turret and/or acrane arm configured to rotate around vertical and/or horizontal axes).Some robotic devices may be configured to operate in more than onespatial dimension orientation so that their components may change theiroperational axis (e.g., with respect to vertical direction) based on theorientation of the robot platform. Such modifications may be effectuatedby an end user of the robot.

SUMMARY

One aspect of the disclosure relates to a non-transitory computerreadable medium having instructions embodied thereon. The instructionsare executable to perform a method for controlling a robotic platform.The method may comprise establishing a data connection to a roboticdevice; receiving information related to a phenotype of the roboticdevice; and issuing a command to a user interface apparatus, the userinterface apparatus executing an action based on the command, thecommand indicative of at least one configuration associated with theinformation. The user interface apparatus may comprise a displayapparatus comprising at least one control configured to relay user inputto the robotic device. Executing the action may cause the user interfaceapparatus to alter a representation of the at least one controlconsistent with the information.

In some implementations, the command may be configured to be issuedautomatically absent an explicit request by the user.

In some implementations, the phenotype may be characterized by one orboth of (i) a hardware configuration of the robotic device or (ii) anoperational configuration of the robotic device. The information may bebased on a statistical parameter related to a plurality of actionsexecuted by the robot responsive to a plurality of user commands relayedby the control. Individual ones of the plurality of actions may beconfigured based on at least one of the hardware configuration theoperational configuration of the robotic device.

In some implementations, the robotic device may comprise at least oneactuator characterized by an axis of motion. The information may beconfigured to relate an orientation of the axis of motion with respectto a reference orientation.

In some implementations, the reference orientation may comprise ageographical coordinate. The information may comprise a computer designfile of the robotic device. The design file may comprise a descriptionof the actuator and the axis of motion.

In some implementations, the reference orientation may comprise an axisof the robotic device. The display apparatus may be characterized by adefault orientation. Altering the representation of the at least onecontrol consistent with the information may comprise: determining anangle between the reference orientation and the axis of motion; andpositioning the at least one control on the display apparatus at theangle relative the default orientation.

In some implementations, the robotic device may comprise first andsecond actuators configured to displace at least a portion of therobotic device in a first direction and a second direction,respectively. The information may comprise parameters of the firstdirection and the second direction. The at least one control maycomprise a first linear motion control and a second linear motioncontrol associated with the first actuator and the second actuator,respectively. The act of altering the representation of the at least onecontrol consistent with the information may comprise: positioning thefirst linear motion control at the first direction; and positioning thesecond linear motion control at the second direction, the seconddirection being perpendicular to the first direction.

In some implementations, the robotic device may be characterized ashaving a default orientation. The first direction and the seconddirection may comprise a direction of longitudinal and transversemotions relative to the default orientation.

In some implementations, the robotic device may be characterized ashaving a default orientation. The robotic device may comprise a firstactuator configured to rotate at least a portion of the robotic devicearound first axis configured vertically with respect to the defaultorientation. A second actuator may be configured to move the roboticdevice in a longitudinal direction relative the default orientation. Theact of altering the representation of the at least one controlconsistent with the information may comprise: positioning the firstlinear motion control along the first direction; and positioning thesecond linear motion control along the second direction, the seconddirection being perpendicular to the first direction.

In some implementations, the first control may comprise a knob having anaxis of rotation configured such that mutual orientation of the axis ofrotation and the default direction matches mutual orientation of thefirst axis and the default orientation of the robotic device.

In some implementations, the at least one control may comprise: a firstslider configured to relate forward and reverse motion commands; and asecond slider configured to relate left and right turn commands, andreverse motion commands. Main axes of the first and the second slidersmay be disposed perpendicular with one another.

In some implementations, the phenotype may be characterized by one orboth of (i) hardware configuration of the robotic device or (ii)operational configuration of the robotic device. The information may beconfigured to relate modification of one or both of the hardwareconfiguration or the operational configuration of the robotic device.

In some implementations, the hardware configuration of the roboticdevice may comprise one or more of a number of motor actuators, arotation axis orientation for individual actuators, or a number ofactuators configured to be activated simultaneously.

In some implementations, the operational configuration of the roboticdevice may comprise one or more of a number of motor actuators, arotation axis orientation for individual actuators, or a number ofactuators configured to be activated simultaneously.

Another aspect of the disclosure relates to a remote control apparatusof a robot. The apparatus may comprise a processor, at least one remotecommunications interface, and a user interface. The at least one remotecommunications interface may be configured to: establish an operativelink to the robot; and communicate to the processor one or moreconfiguration indicators associated with a component of the robot. Theuser interface may be configured to: based on a receipt of theconfiguration indicator, display one or more human perceptible controlelements consistent with a characteristic of the component.

In some implementations, the user interface may comprise a displayapparatus. The component may comprise one or both of a wheel or a joint,characterized by axis of rotation. The characteristic of the componentmay be configured to describe placement of the axis with respect to areference direction. The displaying of one or more human perceptiblecontrol elements consistent with the characteristic may comprisedisposing the control element on the display apparatus at an orientationmatching the placement of the axis with respect to the reference.

In some implementations, the robot may comprise a sensor configured todetermine the placement of the axis with respect to the reference. Therobot may detect and communicate an operation configuration of one ofits elements.

In some implementations, the user interface may comprise one or more oftouch-sensing interface, a contactless motion sensing interface, or aradio frequency wireless interface.

Yet another aspect of the disclosure relates to a method ofcommunicating a robot operational characteristic. The method maycomprise: configuring the robot to detect the operationalcharacteristic; and enabling communication by the robot of theoperational characteristic. The communication of the operationalcharacteristic may be configured to cause adaptation of the userinterface device configured to operate the robot.

In some implementations, the robot may comprise an operational elementcomprising at least one of a wheel or a joint, characterized by an axisof rotation. The operational characteristic may comprise an angle of theaxis relative a reference direction. The adaptation may comprisedisposing a control element associated with the operational element atthe angle and/or displacement relative to the reference on the userinterface device.

In some implementations, the method may comprise: configuring the robotto detect a modification of the operational characteristic; and,responsive to detected modification of the operational characteristic,communicating the modified operational characteristic associated withthe operational element. The communication of the modified operationalcharacteristic may be configured to cause modification of the controlelement consistent with the modified operational characteristic.

In some implementations, the modification of the operationalcharacteristic may comprise a change of the angle and/or displacement byan adjustment amount. The modification of the control element consistentwith the modified operational characteristic may comprise adjustment ofthe disposed control element by the adjustment amount.

In some implementations, the modification of the operationalcharacteristic may comprise a change of the angle and/or thedisplacement by an adjustment amount. The modification of the controlelement consistent with the modified operational characteristic maycomprise adjustment of the disposed control element by the adjustmentamount.

In some implementations, the robot may comprise a humanoid robotcomprising a first joint configured to be rotated with respect to afirst axis and a second joint configured to be rotated with respect tosecond axis. The first and the second axes may be disposed at a non-zeroangle relative to one another. The adaptation of the user interfacedevice may be configured to dispose a first control element and a secondcontrol element adapted to control the first joint and the second joint,respectively, at the angle with respect to one another.

In some implementations, the humanoid robot may comprise a roboticapparatus with its body shape built to resemble that of the human body.

These and other features, and characteristics of the present disclosure,as well as the methods of operation and functions of the relatedelements of structure and the combination of parts and economies ofmanufacture, will become more apparent upon consideration of thefollowing description and the appended claims with reference to theaccompanying drawings, all of which form a part of this specification,wherein like reference numerals designate corresponding parts in thevarious figures. It is to be expressly understood, however, that thedrawings are for the purpose of illustration and description only andare not intended as a definition of the limits of the disclosure. Asused in the specification and in the claims, the singular form of “a”,“an”, and “the” include plural referents unless the context clearlydictates otherwise.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a robotic apparatus, according toone or more implementations.

FIG. 2A is a functional block diagram illustrating a controller for arobotic apparatus, according to one or more implementations.

FIG. 2B is a functional block diagram illustrating a robotic roverapparatus comprising two drive wheels and a controller associatedtherewith, according to one or more implementations.

FIG. 2C is a functional block diagram illustrating a robotic roverapparatus, comprising two articulated drive wheels, and a controllerassociated therewith, according to one or more implementations.

FIG. 3 is a functional block diagram illustrating a user interfacecontroller configured in accordance with operational characteristics ofthe robotic rover apparatus comprising two articulated drive wheels,according to one or more implementations.

FIG. 4A is a graphical illustration depicting a robotic apparatuscapable of operating in 3 spatial dimensions, according to one or moreimplementations.

FIG. 4B is a graphical illustration depicting user interface controllersconfigured of the apparatus of FIG. 4A, according to one or moreimplementations.

FIG. 4C is a graphical illustration depicting user interface controllersconfigured of the apparatus of FIG. 4A, according to one or moreimplementations.

FIG. 5A is a functional block diagram illustrating adaptiveconfiguration of a robot controller responsive to a change of robotoperational configuration, according to one or more implementations.

FIG. 5B is a functional block diagram illustrating adaptiveconfiguration of a robot controller responsive to a change of robotoperational configuration, according to one or more implementations.

FIG. 6 is a functional block diagram depicting a computerized systemconfigured for adaptive configuration of robotic controller, accordingto one or more implementations.

FIG. 7 is a logical flow diagram illustrating a method of operating anadaptive robotic device, in accordance with one or more implementations.

FIG. 8A is a logical flow diagram illustrating a method of adaptingremote controller of a robot based on a change of robot configuration,in accordance with one or more implementations.

FIG. 8B is a logical flow diagram illustrating a method of adaptingremote controller of a robot based on a change of operating environment,in accordance with one or more implementations.

FIG. 9 is a logical flow diagram illustrating a method of training arobotic apparatus using an adaptive remoter controller apparatus, inaccordance with one or more implementations.

All Figures disclosed herein are © Copyright 2013 Brain Corporation. Allrights reserved.

DETAILED DESCRIPTION

Implementations of the present technology will now be described indetail with reference to the drawings, which are provided asillustrative examples so as to enable those skilled in the art topractice the technology. Notably, the figures and examples below are notmeant to limit the scope of the present disclosure to a singleimplementation, but other implementations are possible by way ofinterchange of or combination with some or all of the described orillustrated elements. Wherever convenient, the same reference numberswill be used throughout the drawings to refer to same or like parts.

Where certain elements of these implementations can be partially orfully implemented using known components, only those portions of suchknown components that are necessary for an understanding of the presenttechnology will be described, and detailed descriptions of otherportions of such known components will be omitted so as not to obscurethe disclosure.

In the present specification, an implementation showing a singularcomponent should not be considered limiting; rather, the disclosure isintended to encompass other implementations including a plurality of thesame component, and vice-versa, unless explicitly stated otherwiseherein.

Further, the present disclosure encompasses present and future knownequivalents to the components referred to herein by way of illustration.

As used herein, the term “bus” is meant generally to denote all types ofinterconnection or communication architecture that is used to access thesynaptic and neuron memory. The “bus” may be optical, wireless,infrared, and/or another type of communication medium. The exacttopology of the bus could be for example standard “bus”, hierarchicalbus, network-on-chip, address-event-representation (AER) connection,and/or other type of communication topology used for accessing, e.g.,different memories in pulse-based system.

As used herein, the terms “computer”, “computing device”, and“computerized device” may include one or more of personal computers(PCs) and/or minicomputers (e.g., desktop, laptop, and/or other PCs),mainframe computers, workstations, servers, personal digital assistants(PDAs), handheld computers, embedded computers, programmable logicdevices, personal communicators, tablet computers, portable navigationaids, J2ME equipped devices, cellular telephones, smart phones, personalintegrated communication and/or entertainment devices, and/or any otherdevice capable of executing a set of instructions and processing anincoming data signal.

As used herein, the term “computer program” or “software” may includeany sequence of human and/or machine cognizable steps which perform afunction. Such program may be rendered in a programming language and/orenvironment including one or more of C/C++, C#, Fortran, COBOL, MATLAB™,PASCAL, Python, assembly language, markup languages (e.g., HTML, SGML,XML, VoXML), object-oriented environments (e.g., Common Object RequestBroker Architecture (CORBA)), Java™ (e.g., J2ME, Java Beans), BinaryRuntime Environment (e.g., BREW), and/or other programming languagesand/or environments.

As used herein, the terms “connection”, “link”, “transmission channel”,“delay line”, “wireless” may include a causal link between any two ormore entities (whether physical or logical/virtual), which may enableinformation exchange between the entities.

As used herein, the term “memory” may include an integrated circuitand/or other storage device adapted for storing digital data. By way ofnon-limiting example, memory may include one or more of ROM, PROM,EEPROM, DRAM, Mobile DRAM, SDRAM, DDR/2 SDRAM, EDO/FPMS, RLDRAM, SRAM,“flash” memory (e.g., NAND/NOR), memristor memory, PSRAM, and/or othertypes of memory.

As used herein, the terms “integrated circuit”, “chip”, and “IC” aremeant to refer to an electronic circuit manufactured by the patterneddiffusion of trace elements into the surface of a thin substrate ofsemiconductor material. By way of non-limiting example, integratedcircuits may include field programmable gate arrays (e.g., FPGAs), aprogrammable logic device (PLD), reconfigurable computer fabrics (RCFs),application-specific integrated circuits (ASICs), and/or other types ofintegrated circuits.

As used herein, the terms “microprocessor” and “digital processor” aremeant generally to include digital processing devices. By way ofnon-limiting example, digital processing devices may include one or moreof digital signal processors (DSPs), reduced instruction set computers(RISC), general-purpose (CISC) processors, microprocessors, gate arrays(e.g., field programmable gate arrays (FPGAs)), PLDs, reconfigurablecomputer fabrics (RCFs), array processors, secure microprocessors,application-specific integrated circuits (ASICs), and/or other digitalprocessing devices. Such digital processors may be contained on a singleunitary IC die, or distributed across multiple components.

As used herein, the term “network interface” refers to any signal, data,and/or software interface with a component, network, and/or process. Byway of non-limiting example, a network interface may include one or moreof FireWire (e.g., FW400, FW800, etc.), USB (e.g., USB2), Ethernet(e.g., 10/100, 10/100/1000 (Gigabit Ethernet), 10-Gig-E, etc.), MoCA,Coaxsys (e.g., TVnet™), radio frequency tuner (e.g., in-band or OOB,cable modem, etc.), Wi-Fi (802.11), WiMAX (802.16), PAN (e.g., 802.15),cellular (e.g., 3G, LTE/LTE-A/TD-LTE, GSM, etc.), IrDA families, and/orother network interfaces.

As used herein, the term “Wi-Fi” includes one or more of IEEE-Std.802.11, variants of IEEE-Std. 802.11, standards related to IEEE-Std.802.11 (e.g., 802.11 a/b/g/n/s/v), and/or other wireless standards.

As used herein, the term “wireless” means any wireless signal, data,communication, and/or other wireless interface. By way of non-limitingexample, a wireless interface may include one or more of Wi-Fi,Bluetooth, 3G (3GPP/3GPP2), HSDPA/HSUPA, TDMA, CDMA (e.g., IS-95A,WCDMA, etc.), FHSS, DSSS, GSM, PAN/802.15, WiMAX (802.16), 802.20,narrowband/FDMA, OFDM, PCS/DCS, LTE/LTE-A/TD-LTE, analog cellular, CDPD,satellite systems, millimeter wave or microwave systems, acoustic,infrared (i.e., IrDA), and/or other wireless interfaces.

FIG. 1 illustrates one implementation of an adaptive robotic apparatusfor use with the robot training methodology described hereinafter. Theapparatus 100 of FIG. 1 may comprise an adaptive controller 102 and aplant (e.g., robotic platform) 110. The controller 102 may be configuredto generate control output 108 for the plant 110. The output 108 maycomprise one or more motor commands (e.g., pan camera to the right),sensor acquisition parameters (e.g., use high resolution camera mode),commands to the wheels, arms, and/or other actuators on the robot,and/or other parameters. The output 108 may be configured by thecontroller 102 based on one or more sensory inputs 106. The input 106may comprise data used for solving a particular control task. In one ormore implementations, such as those involving a robotic arm orautonomous robot, the signal 106 may comprise a stream of raw sensordata and/or preprocessed data. Raw sensor data may include dataconveying information associated with one or more of proximity,inertial, terrain imaging, and/or other information. Preprocessed datamay include data conveying information associated with one or more ofvelocity, information extracted from accelerometers, distance toobstacle, positions, and/or other information. In some implementations,such as those involving object recognition, the signal 106 may comprisean array of pixel values in the input image, or preprocessed data. Pixeldata may include data conveying information associated with one or moreof RGB, CMYK, HSV, HSL, grayscale, and/or other information.Preprocessed data may include data conveying information associated withone or more of levels of activations of Gabor filters for facerecognition, contours, and/or other information. In one or moreimplementations, the input signal 106 may comprise a target motiontrajectory. The motion trajectory may be used to predict a future stateof the robot on the basis of a current state and the target state. Inone or more implementations, the signals in FIG. 1 may be encoded asspikes, as described in detail in U.S. patent application Ser. No.13/842,530 entitled “ADAPTIVE PREDICTOR APPARATUS AND METHODS”, filedMar. 15, 2013, incorporated supra.

The controller 102 may be operable in accordance with a learning process(e.g., reinforcement learning and/or supervised learning). In one ormore implementations, the controller 102 may optimize performance (e.g.,performance of the system 100 of FIG. 1) by minimizing average value ofa performance function as described in detail in co-owned U.S. patentapplication Ser. No. 13/487,533, entitled “STOCHASTIC SPIKING NETWORKLEARNING APPARATUS AND METHODS”, incorporated herein by reference in itsentirety.

Learning process of adaptive controller (e.g., 102 of FIG. 1) may beimplemented using a variety of methodologies. In some implementations,the controller 102 may comprise an artificial neuron network e.g., thespiking neuron network described in U.S. patent application Ser. No.13/487,533, entitled “STOCHASTIC SPIKING NETWORK LEARNING APPARATUS ANDMETHODS”, filed Jun. 4, 2012, incorporated supra, configured to control,for example, a robotic rover.

Individual spiking neurons may be characterized by internal state. Theinternal state may, for example, comprise a membrane voltage of theneuron, conductance of the membrane, and/or other parameters. The neuronprocess may be characterized by one or more learning parameters, whichmay comprise input connection efficacy, output connection efficacy,training input connection efficacy, response generating (firing)threshold, resting potential of the neuron, and/or other parameters. Inone or more implementations, some learning parameters may compriseprobabilities of signal transmission between the units (e.g., neurons)of the network.

In some implementations, the training input (e.g., 104 in FIG. 1) may bedifferentiated from sensory inputs (e.g., inputs 106) as follows. Duringlearning, data (e.g., spike events) arriving to neurons of the networkvia input 106 may cause changes in the neuron state (e.g., increaseneuron membrane potential and/or other parameters). Changes in theneuron state may cause the neuron to generate a response (e.g., output aspike). Teaching data arriving to neurons of the network may cause (i)changes in the neuron dynamic model (e.g., modify parameters a,b,c,d ofIzhikevich neuron model, described for example in co-owned U.S. patentapplication Ser. No. 13/623,842, entitled “SPIKING NEURON NETWORKADAPTIVE CONTROL APPARATUS AND METHODS”, filed Sep. 20, 2012,incorporated herein by reference in its entirety); and/or (ii)modification of connection efficacy, based, for example, on timing ofinput spikes, teacher spikes, and/or output spikes. In someimplementations, teaching data may trigger neuron output in order tofacilitate learning. In some implementations, teaching signal may becommunicated to other components of the control system.

During operation (e.g., subsequent to learning), data (e.g., spikeevents) arriving to neurons of the network may cause changes in theneuron state (e.g., increase neuron membrane potential and/or otherparameters). Changes in the neuron state may cause the neuron togenerate a response (e.g., output a spike). Teaching data may be absentduring operation, while input data are required for the neuron togenerate output.

In one or more implementations, such as object recognition and/orobstacle avoidance, the input 106 may comprise a stream of pixel valuesassociated with one or more digital images. In one or moreimplementations (e.g., video, radar, sonography, x-ray, magneticresonance imaging, and/or other types of sensing), the input maycomprise electromagnetic waves (e.g., visible light, IR, UV, and/orother types of electromagnetic waves) entering an imaging sensor array.In some implementations, the imaging sensor array may comprise one ormore of RGCs, a charge coupled device (CCD), an active-pixel sensor(APS), and/or other sensors. The input signal may comprise a sequence ofimages and/or image frames. The sequence of images and/or image framemay be received from a CCD camera via a receiver apparatus and/ordownloaded from a file. The image may comprise a two-dimensional matrixof RGB values refreshed at a 25 Hz frame rate. It will be appreciated bythose skilled in the arts that the above image parameters are merelyexemplary, and many other image representations (e.g., bitmap, CMYK,HSV, HSL, grayscale, and/or other representations) and/or frame ratesare equally useful with the present technology. Pixels and/or groups ofpixels associated with objects and/or features in the input frames maybe encoded using, for example, latency encoding described in U.S. patentapplication Ser. No. 12/869,583, filed Aug. 26, 2010 and entitled“INVARIANT PULSE LATENCY CODING SYSTEMS AND METHODS”; U.S. Pat. No.8,315,305, issued Nov. 20, 2012, entitled “SYSTEMS AND METHODS FORINVARIANT PULSE LATENCY CODING”; U.S. patent application Ser. No.13/152,084, filed Jun. 2, 2011, entitled “APPARATUS AND METHODS FORPULSE-CODE INVARIANT OBJECT RECOGNITION”; and/or latency encodingcomprising a temporal winner take all mechanism described U.S. patentapplication Ser. No. 13/757,607, filed Feb. 1, 2013 and entitled“TEMPORAL WINNER TAKES ALL SPIKING NEURON NETWORK SENSORY PROCESSINGAPPARATUS AND METHODS”, each of the foregoing being incorporated hereinby reference in its entirety.

In one or more implementations, object recognition and/or classificationmay be implemented using spiking neuron classifier comprisingconditionally independent subsets as described in co-owned U.S. patentapplication Ser. No. 13/756,372 filed Jan. 31, 2013, and entitled“SPIKING NEURON CLASSIFIER APPARATUS AND METHODS” and/or co-owned U.S.patent application Ser. No. 13/756,382 filed Jan. 31, 2013, and entitled“REDUCED LATENCY SPIKING NEURON CLASSIFIER APPARATUS AND METHODS”, eachof the foregoing being incorporated herein by reference in its entirety.

In one or more implementations, encoding may comprise adaptiveadjustment of neuron parameters, such neuron excitability described inU.S. patent application Ser. No. 13/623,820 entitled “APPARATUS ANDMETHODS FOR ENCODING OF SENSORY DATA USING ARTIFICIAL SPIKING NEURONS”,filed Sep. 20, 2012, the foregoing being incorporated herein byreference in its entirety.

In some implementations, analog inputs may be converted into spikesusing, for example, kernel expansion techniques described in co pendingU.S. patent application Ser. No. 13/623,842 filed Sep. 20, 2012, andentitled “SPIKING NEURON NETWORK ADAPTIVE CONTROL APPARATUS ANDMETHODS”, the foregoing being incorporated herein by reference in itsentirety. In one or more implementations, analog and/or spiking inputsmay be processed by mixed signal spiking neurons, such as U.S. patentapplication Ser. No. 13/313,826 entitled “APPARATUS AND METHODS FORIMPLEMENTING LEARNING FOR ANALOG AND SPIKING SIGNALS IN ARTIFICIALNEURAL NETWORKS”, filed Dec. 7, 2011, and/or co-pending U.S. patentapplication Ser. No. 13/761,090 entitled “APPARATUS AND METHODS FORIMPLEMENTING LEARNING FOR ANALOG AND SPIKING SIGNALS IN ARTIFICIALNEURAL NETWORKS”, filed Feb. 6, 2013, each of the foregoing beingincorporated herein by reference in its entirety.

The rules may be configured to implement synaptic plasticity in thenetwork. In some implementations, the plastic rules may comprise one ormore spike-timing dependent plasticity, such as rule comprising feedbackdescribed in co-owned and co-pending U.S. patent application Ser. No.13/465,903 entitled “SENSORY INPUT PROCESSING APPARATUS IN A SPIKINGNEURAL NETWORK”, filed May 7, 2012; rules configured to modify of feedforward plasticity due to activity of neighboring neurons, described inco-owned U.S. patent application Ser. No. 13/488,106, entitled “SPIKINGNEURON NETWORK APPARATUS AND METHODS”, filed Jun. 4, 2012; conditionalplasticity rules described in U.S. patent application Ser. No.13/541,531, entitled “CONDITIONAL PLASTICITY SPIKING NEURON NETWORKAPPARATUS AND METHODS”, filed Jul. 3, 2012; plasticity configured tostabilize neuron response rate as described in U.S. patent applicationSer. No. 13/691,554, entitled “RATE STABILIZATION THROUGH PLASTICITY INSPIKING NEURON NETWORK”, filed Nov. 30, 2012; activity-based plasticityrules described in co-owned U.S. patent application Ser. No. 13/660,967,entitled “APPARATUS AND METHODS FOR ACTIVITY-BASED PLASTICITY IN ASPIKING NEURON NETWORK”, filed Oct. 25, 2012, U.S. patent applicationSer. No. 13/660,945, entitled “MODULATED PLASTICITY APPARATUS ANDMETHODS FOR SPIKING NEURON NETWORKS”, filed Oct. 25, 2012; and U.S.patent application Ser. No. 13/774,934, entitled “APPARATUS AND METHODSFOR RATE-MODULATED PLASTICITY IN A SPIKING NEURON NETWORK”, filed Feb.22, 2013; multi-modal rules described in U.S. patent application Ser.No. 13/763,005, entitled “SPIKING NETWORK APPARATUS AND METHOD WITHBIMODAL SPIKE-TIMING DEPENDENT PLASTICITY”, filed Feb. 8, 2013, each ofthe foregoing being incorporated herein by reference in its entirety.

In one or more implementations, neuron operation may be configured basedon one or more inhibitory connections providing input configured todelay and/or depress response generation by the neuron, as described inU.S. patent application Ser. No. 13/660,923, entitled “ADAPTIVEPLASTICITY APPARATUS AND METHODS FOR SPIKING NEURON NETWORK”, filed Oct.25, 2012, the foregoing being incorporated herein by reference in itsentirety

Connection efficacy updated may be effectuated using a variety ofapplicable methodologies such as, for example, event based updatesdescribed in detail in co-owned U.S. patent application Ser. No.13/239,255 , filed Sep. 21, 2011, entitled “APPARATUS AND METHODS FORSYNAPTIC UPDATE IN A PULSE-CODED NETWORK”; 201220, U.S. patentapplication Ser. No. 13/588,774, entitled “APPARATUS AND METHODS FORIMPLEMENTING EVENT-BASED UPDATES IN SPIKING NEURON NETWORK”, filed Aug.17, 2012; and U.S. patent application Ser. No. 13/560,891 entitled“APPARATUS AND METHODS FOR EFFICIENT UPDATES IN SPIKING NEURONNETWORKS”, each of the foregoing being incorporated herein by referencein its entirety.

A neuron process may comprise one or more learning rules configured toadjust neuron state and/or generate neuron output in accordance withneuron inputs.

In some implementations, the one or more learning rules may comprisestate dependent learning rules described, for example, in U.S. patentapplication Ser. No. 13/560,902, entitled “APPARATUS AND METHODS FORSTATE-DEPENDENT LEARNING IN SPIKING NEURON NETWORKS”, filed Jul. 27,2012 and/or pending U.S. patent application Ser. No. 13/722,769 filedDec. 20, 2012, and entitled “APPARATUS AND METHODS FOR STATE-DEPENDENTLEARNING IN SPIKING NEURON NETWORKS”, each of the foregoing beingincorporated herein by reference in its entirety.

In one or more implementations, the one or more leaning rules may beconfigured to comprise one or more reinforcement learning, unsupervisedlearning, and/or supervised learning as described in co-owned andco-pending U.S. patent application Ser. No. 13/487,499 entitled“STOCHASTIC APPARATUS AND METHODS FOR IMPLEMENTING GENERALIZED LEARNINGRULES, incorporated supra.

In one or more implementations, the one or more leaning rules may beconfigured in accordance with focused exploration rules such asdescribed, for example, in U.S. patent application Ser. No. 13/489,280entitled “APPARATUS AND METHODS FOR REINFORCEMENT LEARNING IN ARTIFICIALNEURAL NETWORKS”, filed Jun. 5, 2012, the foregoing being incorporatedherein by reference in its entirety.

Adaptive controller (e.g., the controller apparatus 102 of FIG. 1) maycomprise an adaptable predictor block configured to, inter alia, predictcontrol signal (e.g., 108) based on the sensory input (e.g., 106 inFIG. 1) and teaching input (e.g., 104 in FIG. 1) as described in, forexample, U.S. patent application Ser. No. 13/842,530 entitled “ADAPTIVEPREDICTOR APPARATUS AND METHODS”, filed Mar. 15, 2013, incorporatedsupra.

Robotic devices (e.g., plant 110 of FIG. 1) may comprise components thatmay enable the robot to perform actions in 1, 2, and/or 3-dimensionalspace. Some robotic devices may comprise one or more components that maybe configured to operate in more than one spatial dimension (e.g., aturret, and/or a crane arm). Such components may be configured to rotatearound vertical and/or horizontal axes. Such configurations may beeffectuated by a user of the robot, e.g., when assembling a robot usingLEGO® Mindstorms kit. A robot may be trained by a user and/or trainerusing, e.g., robot training methodology.

It may be beneficial to train and/or operate robotic devices using aremote control device. One implementation of a computerized controllerapparatus configured for remote control a robotic devices is illustratedin FIG. 2A. The remote control apparatus 260 may comprise memory 264.Processing capacity 266 may be available for other hardware, firmware,and/or software needs of the controller device. The apparatus maycomprise display 262 configured to represent control elements to theuser. The apparatus may comprise input sensor 270 configured to receiveuser input. In one or more implementations, the display and the sensorfunctionality may be combined by a single physical block (e.g., a touchscreen display). In some implementations, the display and the inputsensor portions may be implemented separate from one another. In one ormore implementations, the input sensor may comprise one or more ofhaptic sensor (e.g., pressure, capacitive, resistive touch sensor),light sensor (e.g., camera), audio sensor, electromagnetic sensor (e.g.,infrared and/or radio frequency), vibrational sensor, ultrasonic sensor,temperature sensor, radar sensor, lidar sensor, sonar sensor, and/orother sensors. In one or more implementations, a user may providecommands to the controller via one or more of touch, speech, audio(e.g., clicks, claps, whistles, and/or other sounds), gestures, eyedirection, and/or other types of input.

The display 262 may comprise any of a liquid crystal display (LCD),light emitting diode (LED), MEMS micro-shutter, interferometricmodulator displays (IMOD), carbon nanotube-based displays, digital lightprojection, and/or other applicable hardware display implementations.

The device 260 may comprise a mechanical platform (e.g., enclosureand/or frame), platform sensor 268, electrical components 272, powercomponents 274, network interface 276, and/or other components. In someimplementations, the platform sensor 268 may comprise a position sensorand/or an orientation sensor configured to determine location and/ororientation of the remote control 26 relative a reference (e.g.,geographical reference and/or robot frame reference). Consistent withthe present disclosure, the various components of the device may beremotely disposed from one another, and/or aggregated. For example,processing (e.g., user input recognition) may be performed by a remoteserver apparatus, and the processed data (e.g., user commands) may becommunicated to the remote controller via the network interface 276.

The electrical components 272 may include virtually any electricaldevice for interaction and manipulation of the outside world. This mayinclude, without limitation, light/radiation generating devices (e.g.LEDs, IR sources, light bulbs, and/or other devices), audio devices,monitors/displays, switches, heaters, coolers, ultrasound transducers,lasers, and/or other electrical devices. These devices may enable a widearray of applications for the robotic apparatus in industrial, hobbyist,building management, medical device, military/intelligence, and/or otherfields (as discussed below).

The network interface may include one or more connections to externalcomputerized devices to allow for, inter alia, management of the roboticdevice. The connections may include any of the wireless and/or wire-lineinterfaces discussed above. The connections may include customizedand/or proprietary connections for specific applications.

The power system 274 may be tailored to the needs of the application ofthe device. For example, for some implementations, a wireless powersolution (e.g. battery, solar cell, inductive (contactless) powersource, rectification, and/or other wireless power solution) may beappropriate. For other implementations, however, battery backup and/ordirect wall power may be superior.

Various realizations the remote control apparatus 260 may be envisaged,such as, for example, a tablet computer, a smartphone, a portablecomputer (laptop), and/or other device comprising a display and a userinput interface. In one or more implementations, the user inputinterface may comprise one of touch sensor, sound sensor, proximitysensor, visual sensor, and/or other sensor.

The remote control apparatus may be configured to interface to a roboticdevice (e.g., rover 200 of FIG. 2B). The display 262 may be configuredto display one or more controls corresponding to one or more componentsof the robotic device. In one or more implementations, the controllablecomponents of the robotic device may include virtually any type ofdevice capable of motion or performance of a desired function or task.These may include, without limitation, motors, servos, pumps,hydraulics, pneumatics, stepper motors, rotational plates,micro-electro-mechanical devices (MEMS), electroactive polymers, arms,actuators, legs, and/or other components. The controller 260 may beutilized to enable physical interaction and manipulation of thesecomponents.

In some implementations, particularly wherein the robotic device maycomprise multiple controllable components (e.g., two wheels 206, 208 inFIG. 2B), the remote control interface may be adapted to presentindividual controls corresponding to one or more individual componentsof the robotic plant. As shown in FIG. 2B, the remote control display212 may comprise slider control 214 operable along direction shown byarrow 218. The 214 may be utilized by a use to operate motor configuredto rotate the wheels 206, 204 of the rover 206 along directionillustrated by arrow 208.

FIG. 2C illustrates a robotic rover 220 comprising a pair of fixed-anglewheels 222, 224 and a pair of articulating wheels 226, 228. Generallyspeaking, fixed-angle wheels may describe wheels that rotate about afixed axis and do not pivot (e.g., the rear wheels of a car).Articulating wheels may describe wheels that rotate about a pivotableaxis (e.g., the front wheels of a car). The fixed angle wheels may beconfigured to propel the rover 220 along direction denoted by arrow 232.The articulated wheels 228 may be coupled to a motor actuator configuredto rotate the wheels 226, 228 by angle 230 thereby enabling roverdisplacement in a direction denoted by arrow 234. A remote controller240 associated with the rover implantation 220 may comprise a pair ofslide control elements 244, 242 configured to be moved by a user alongdirection 246. In one or more implementations, the slider 242 may beutilized to control the rear wheels 222, 224, thereby controlling roverforward/back movement 232; the slider 244 may be utilized to controlangle 228 of the front wheels 226, 228, thereby controlling roverleft/right movement 232.

In some implementations, the remote control realization 240 may beutilized with a rover configuration (not shown) comprising fourmotorized wheels configured to be controlled in pairs (e.g., front andback right wheels, and front and back left wheels).

While motion of the control element configured to control forward/backmotion may match the direction of the rover movement associated with theslider 242 (e.g., arrows 232 and 246 are parallel with one another),motion of the control element configured to control left/right rovermovement (e.g., slider 244) may not match the direction of the rovermovement (e.g., arrows 234 and 246 are perpendicular with one another).Some implementations may include a remote control configured such thatconfiguration of control elements (e.g., sliders 242, 244) matches thedirection of respective robot motions (e.g., 232, 234, respectively).

FIG. 3 illustrates a robotic controller configured consistent withphenotype of the robot being controlled. The term phenotype may describean operational and/or hardware configuration of the robot. By way ofillustration, two robots comprising two legs and four legs may bereferred to as comprising different phenotype; two robots, eachcomprising manipulator configured to be rotated around two differentaxes (e.g., oriented at a non-zero angle with respect to one another)may be referred to as comprising different phenotypes.

The controller 300 may be configured to operate a robotic deviceconfigured to move in a plane characterized by two orthogonal motioncomponents (e.g., rover 220 configured to move on a horizontal planecharacterized by components 232, 234). The controller apparatus 300 maycomprise sliders 302, 304 disposed such that direction of their movement306, 308, respectively, matches motion components of the rover (e.g.,components 232, 234 of the rover 220). In one or more implementations,the control element 302 may be referred to as the speed control(throttle). The control element 304 may be referred to as directioncontrol (steering.)

In some implementations, steering control elements of the controllerapparatus configured to control motion of a rover in two dimensions maycomprise a knob, e.g., 324 of the controller 320 in FIG. 3. Turning ofthe knob indicated by the curve 328 may enable control of the rovermotion direction component (e.g., component 234 of the rover 220).

In one or more implementations, the controller 300 and/or 320 may beutilized to control a rover comprising four motorized wheelscontrollable in pairs (e.g., front and back right wheels, and front andback left wheels).

FIG. 4A illustrates a robotic apparatus comprising multiple controllablecomponents operable in two or more degrees of freedom (DoF). Theapparatus 400 may comprise a platform 402 configured to traverse therobotic apparatus 400 in a direction indicated by arrow 424. In someimplementations, the platform may comprise on or more motor drivenwheels (e.g., 404) and/or articulated wheels.

The platform may be adapted to accept a telescopic arm 410 disposedthereupon. The arm 410 may comprise one or more portions (e.g., boom412, portion 414) configured to be moved in directions shown by arrows406 (telescope boom 412 in/out); 422 (rotate the portion 414 up/down). Autility attachment 415 may be coupled to the arm 414. In one or moreimplementations, the attachment 415 may comprise a hook, a graspingdevice, a ball, and/or any applicable attachment. The attachment 415 maybe moved in direction shown by arrow 426. The arm 410 may be configuredto elevate up/down (using for example, motor assembly 411) and/or berotated as shown by arrows 420, 428 respectively in FIG. 4A.

FIG. 4B-4C illustrate robotic controller realizations configuredconsistent with phenotype of the robotic apparatus of FIG. 4A. Thecontroller 450 of FIG. 4B may comprises a plurality of controls elementsadapted to manipulate the platform 402 (e.g., controls 462, 464), andthe arm 410 (e.g., the controls 452, 454, 456, 458). One or more of thecontrols 452, 454, 456, 458, 462, 464, may comprise joystick, sliderand/or another control type (e.g., knob 478 described with respect toFIG. 4C). The control elements in FIGS. 4B-4C may comprise hardwareelements and/or software control rendered, for example, on a touchscreen of a portable computerized device (e.g., smartphone, tablet,and/or other portable device). In some implementations the controlelements may consist of the configuration of a contactless motionsensing interface system, with and/or without an explicit indication asto the current configuration. In the case of an explicit indication, forexample, the configuration may be indicated by representations displayedvia a screen, or via changes on a device held by the user to perform themotions, or via light projected onto the source of motion (e.g., ontothe hands of the human operator).

The control elements 464, 462 may be configured to operate alongdirections 462, 460, respectively, and control two dimensional motion ofthe platform 402 (shown by arrows 424, 429, respectively in FIG. 4A).The control elements 456, 462, 464 may be configured to operate alongdirection 463 and control vertical motion of the attachment 415, the arm410, the and/or boom 412. The control element 458 may be adapted tocontrol the horizontal orientation of the arm 410 (e.g., as shown by thearrow 428 in FIG. 4A). Another control element(s) (not shown) may beused to control the rotation 422 of the portion 414.

In some implementations of the robotic device (e.g., the roboticapparatus 400), the portion 414 may be omitted during deviceconfiguration, and/or configured to extend/telescope in/out. Thecontroller 450 interface may be configured to in accordance withmodification of the robotic device, by for example, providing anadditional control element (not shown) to control the extension of theportion 414. In some implementations in order to reduce number ofcontrols, additional control operations may be effectuated bycontemporaneous motion of two or more control elements. By way ofexample, simultaneous motion of control elements 454, 456 may effectuateextension control of the portion 414.

The controller 457 of FIG. 4C may comprise a plurality of controlselements adapted to manipulate the platform 402 (e.g., controls 482,480), and the arm 410 (e.g., the controls 472, 474, 476, 478). One ormore of the controls 472, 474, 476, 480 may comprise joystick, sliderand/or another linear motion control type. The elements 478, 482 maycomprise rotary motion controls (e.g., knobs) configured to be rotatesas shown by arrows 486, 488, respectively. The control elements in FIGS.4B-4C may comprise hardware elements and/or software control rendered,for example, on a touch screen of a portable computerized device (e.g.,smartphone, tablet).

A remote controller user interface configured in accordance with therobotic phenotype may be referred to as having matching, conforming,and/or compliant configuration. The methodology providing conformingremote controllers may be utilized with robotic devices configurable tooperate in multiple phenotype configurations. In some implementations,multiple phenotype configurations may be effectuated due to areconfiguration and/or replacement of a portion of robotic plant (e.g.,replacing horizontally rotating manipulator arm with a telescopic arm).In one or more implementations, individual ones of multiple phenotypesmay be realized by operating a robot in different orientations, e.g., asillustrated below.

FIGS. 5A-5B illustrate adaptation of a remote control interface based onrobot configuration 500, 520. The robot of configuration 500 may beadapted to move along surface 508 along directions shown by arrow 514using two (or more) wheels 510, 512. In some implementations, the wheels510, 512 may comprise wheel pairs with one or more wheel pairs beingdriven by a motor. The robot may comprises a turret 504 configured torotate around axis 506. The turret 504 may be configured to support asensing and/or projection apparatus such as an antenna, searchlight,camera lens, and/or other device. The direction 514 may correspond tohorizontal surface, the axis 506 may be configured vertical as shown inFIG. 5A.

A robot of configuration 500 may be provided with a remote controllerapparatus 520. User interface of the remote controller 520 may comprisetwo slider control elements 522, 524. The control elements 522, 524 maybe configured to be moved along direction shown by arrow 526. In, someimplementations, displacing the slider 524 along direction 526 may causeleft/right motion (e.g., shown by arrow 514) of the rover 502;displacing the slider 522 along direction 526 may cause left/rightrotation of the turret 504 (e.g., shown by arrow 507).

The rover 502 may be configurable to operate in orientation shown inFIG. 5A and another orientation shown in FIG. 5B. In the configuration530, the robot 502 may be adapted to move along the surface 508 alongdirections shown by the arrow 514 using wheels 510, 536. The robot 502orientation 530 in FIG. 5B may be configured perpendicular to the robotorientation of FIG. 5A. The turret 504 of the robot configuration ofFIG. 5B may rotate (e.g., up/down) with respect to the horizontal axis532. Responsive to a change of rover operational configuration (e.g.,from configuration 500 to 300), user interface 520 may adapt to reflectthe change (e.g., a change of the turret 504 rotation direction).

Panel 540 in FIG. 5B illustrates a remote controlled configured inaccordance with operational configuration 530 of the rover 502, inaccordance with one or more implementations. User interface of theremote controller 540 may comprise two slider control elements 542, 544.The control element 542 may be configured to be moved along directionshown by arrow 546 thereby controlling left/right motion (e.g., shown byarrow 514) of the rover 502 in FIG. 5B. The control element 544 may beconfigured to be moved along direction shown by arrow 548. In someimplementations, displacing the slider 544 along vertical direction maycause up/down motion of the rover turret 504 (e.g., shown by arrow 514)of the rover 502. Displacing the slider 522 along direction 526 maycause left/right rotation of the turret 504 (e.g., shown by arrow 536 inFIG. 5B). Configuring user interface of the remoter controller 540consistent with the operational configuration 530 of the robot mayfacilitate training of robot 502 by a user.

FIG. 6 illustrates an adaptive computerized system configured to enableadaptive configuration of robotic controller, according to one or moreimplementations. The system 600 may comprise a robotic apparatus 602comprising one or more motorized operational elements (e.g., wheel, arm,manipulator, leg, and/or other). The robotic apparatus 602 may beconfigured to operate in more than one spatial dimension and/ororientation. The robotic apparatus 602 may comprise componentsconfigured to change their operational axis (e.g., with respect tovertical direction) based on the orientation of the robot platform. Suchmodifications may be effectuated by an end user of the robot duringoperation and/or assembly.

The robotic apparatus 602 may communicate with a remote controllerdevice 604 via a remote link 612. In one or more implementations, therobotic apparatus 602 may comprise a mobile rover 200, 220, roboticapparatus 400, 502 of FIGS. 2B-2C, 4A, 5, respectively and/or anotherrobot realization (e.g., bi-pedaling humanoid robot). In someimplementations, the remote controller 604 may comprise an adaptablecomputerized user interface, e.g., as described above with respect toFIGS. 2A, 3. The remote interface 612 may comprise any applicable wiredremote interface (USB, Ethernet, Firewire, and/or other wired remoteinterface) and/or wireless remote interface (e.g., radio frequency,light, ultrasonic, and/or other remote interface). The remote controllerapparatus 604 may be configured to receive an operational configuration(e.g., number of joints, number of motor actuators, orientation ofjoints and/or motor actuators, and/or other operation configurations) ofthe robotic apparatus 602. In some implementations, the robot 602 mayprovide (push) the configuration via, e.g., link 612 to the remotecontroller 604. In one or more implementations, the remote controllermay request (poll) an update of the robot 602 configuration via, e.g.,link 612. In one or more implementations, the user may input robotconfiguration information into the remote controller apparatus 604.

In some implementations, the robot 602 may provide or publish theconfiguration via link 614 to a remote computerized device 610. In someimplementations, the computerized device 610 may comprise a cloud serverdepository and/or remote server configured to store operational softwareand/or configuration of the robot 602. In some implementations, theapparatus 610 may be configured to store software code or firmware fordownload to one or more robotic devices (e.g., robotic apparatus 602),for example, the methodology described in U.S. patent application Ser.No. 13/830,398 entitled “NEURAL NETWORK LEARNING AND COLLABORATIONAPPARATUS AND METHODS” (the '398 application), filed Mar. 14, 2013,incorporated herein by reference in its entirety. As described in the'398 application, the cloud server may connect to the robotic apparatus602 (or otherwise accesses information about the apparatus, such as froma network server or cloud database, or other user device) to collecthardware and other data of utility in determining compatibility withavailable software images. In some implementations, the user interfacedevice 604 may collect this information from the robotic apparatus 602and forward it to the server 610. The user interface device 604 mayretrieve configuration of the robotic apparatus 602 from the depository610 via link 608. In one or more implementations, the link 608 maycomprise wired link (e.g., USB, SPI, I2C, Ethernet, Firewire, and/orother wired link) and/or wireless link (e.g., radio frequency, light,ultrasonic, and/or other wireless link). The remote controller apparatusmay utilize updated robot configuration information to configure a userinterface in accordance with the robot operational configuration usingany of the applicable methodologies described herein.

FIGS. 7-9 illustrate methods of training an adaptive apparatus of thedisclosure in accordance with one or more implementations. Theoperations of methods 700, 800, 820, 900 presented below are intended tobe illustrative. In some implementations, methods 700, 800, 820, 900 maybe accomplished with one or more additional operations not described,and/or without one or more of the operations discussed. Additionally,the order in which the operations of methods 700, 800, 820, 900 areillustrated in FIGS. 7-9 described below is not intended to be limiting.

In some implementations, methods 700, 800, 820, 900 may be implementedin one or more processing devices (e.g., a digital processor, an analogprocessor, a digital circuit designed to process information, an analogcircuit designed to process information, a state machine, and/or othermechanisms for electronically processing information). The one or moreprocessing devices may include one or more devices executing some or allof the operations of methods 700, 800, 820, 900 in response toinstructions stored electronically on an electronic storage medium. Theone or more processing devices may include one or more devicesconfigured through hardware, firmware, and/or software to bespecifically designed for execution of one or more of the operations ofmethods 700, 800, 820, 900. Operations of methods 700, 800, 820, 900 maybe utilized with a remote controlled robotic apparatus such asillustrated in FIGS. 2B-6.

At operation 702 of method 700, illustrated in FIG. 7, a configurationof the robotic apparatus may be determined. The configuration maycomprise robot hardware components (e.g., number of joints, number ofmotor actuators, orientation of joints and/or motor actuators, and/orother information associated with a configuration) and or operationalparameters (e.g., orientation of robotic platform and/or otheroperational parameters). In one or more implementations, theconfiguration may be may be polled (e.g., requested) by, for example,using a remote controller apparatus and/or a computerized server,described above with respect to FIG. 6. In some implementations, theconfiguration may be provided or pushed by the robot. In one or moreimplementations, the configuration may be provided by a user based onvisual inspection and/or a documentation review. The configuration maybe loaded from a data depository (e.g., based on robot's serial number).

At operation 704, the configuration may be communicated to theadaptation logic associated with remote control interface. In someimplementations, the adaptation logic may comprise a processor of theremoter controller (e.g., 266 in FIG. 2A). In one or moreimplementations, the adaptation logic may be embodied in a cloud server(e.g., 610 in FIG. 6).

At operation 706, a configuration of a remote controller interface thatis consistent with the robot configuration may be determined. In one ormore implementations, the consistent interface configuration may bebased on disposing one or more control elements (e.g., sliders 542, 544in FIG. 5B) aligned with axes of respective motion components of therobot. In some implementations, the compliant remote interface structuremay be communicated by a remote logic (e.g., server 610) to the remotecontroller.

At operation 708, the robot may be operated using a remote controllercharacterized by the consistent interface configuration.

FIG. 8 illustrates a method of adapting remote controller of a robotbased on a change of robot configuration, in accordance with one or moreimplementations.

At operation 802 interface of controller of a robot may be arranged inaccordance with robot hardware configuration. In some implementations,the robot hardware configuration may comprise one or more of a number ofjoints, a number of motor actuators, orientation of joints and/or motoractuators, and/or other information associated with configuration.Arrangement of the remote control interface may comprise disposingcontrol element (e.g., sliders 302, 304 in FIG. 3) parallel to therespective motion components (e.g., 232, 234 of FIG. 2C).

At operation 804, the robot may be operated using the interfaceconfiguration determined at operation 802. In some implementations,operations 804 may comprise controlling speed and direction of the rover220 of FIG. 2C using sliders 302, 304 and/or the slider 302 and the knob324 of FIG. 3, and/or controlling the robotic apparatus 400 of FIG. 4Ausing control elements shown and described with respect to FIGS. 4B-4Cabove.

At operation 806, a modification of the robot hardware configuration maybe detected. In some implementations, the modification of the robothardware configuration may comprise addition and/or removal of jointsand/or motor actuators, change of orientation of joints and/or motoractuators, coupling and/or decoupling or paired wheels, and/or otherchanges or modifications. In one or more implementations, themodification of the robot hardware configuration may be performed by auser. The modification of the robot hardware configuration may occur dueto a component malfunction (e.g., burned out motor). The detection maybe performed automatically based on a configurations file and/orexecution of a diagnostic process by hardware component controller(e.g., servo error status report). In some implementations, themodification detection information may be provided by a user (e.g., viachanges to a configuration register). In one implementations, themodification may comprise conversion of fixed front wheel vehicle (e.g.,the rover 200 of FIG. 2B) to a vehicle comprising articulated frontwheels (e.g., 220 in FIG. 2C).

At operation 808, an interface of the robotic controller may be adjustedconsistent with the modified robot configuration as described, forexample, with respect to operation 802 above.

At operation 810, the robot may be operated using the adjusted interfaceconfiguration determined at operation 808. In some implementations,operations 810 may comprise controlling speed and direction of the rover220 of FIG. 2C using sliders 302, 304 and/or the slider 302 and the knob324 of FIG. 3 and/or controlling the robotic apparatus 400 of FIG. 4Ausing control elements 452, 458, 462, 464 shown in FIG. 4B and/orcontrol elements 478, 482 shown in FIG. 4C.

FIG. 8B is a logical flow diagram illustrating a method of adapting aremote controller of a robot based on a change of operating environmentand operational parameters of a robot, in accordance with one or moreimplementations.

At operation 822, an interface of a controller of a robot may bearranged in accordance with a robot hardware configuration. In someimplementations, the robot hardware configuration may comprise one ormore of a number of joints, a number of motor actuators, orientation ofjoints and/or actuators, and/or other information associated withconfiguration.

At operation 824, the robot may be operated using the interfaceconfiguration determined at operation 822. In some implementations,operations 824 may comprise controlling speed and/or direction of therover 220 of FIG. 2C using sliders 302, 304 and/or the slider 302 andthe knob 324 of FIG. 3 and/or controlling the robotic apparatus 400 ofFIG. 4A using control elements 452, 458, 462, 464 shown in FIG. 4Band/or control elements 478, 482 shown in FIG. 4C.

At operation 826, changes of the robot operational configuration and/orenvironment characteristics may be detected. In some implementations,changes of the robot operational configuration may be based on a changeof robot orientation (e.g. as described with respect to FIGS. 5A-5B). Inor more implementations, changes of the environment characteristics maycomprise one or more of physical load of the robot, wind and/or otherexternal forces that may be acting upon the robot, energy level and/orpower draw of the robot, terrain characteristics (e.g., smoothnessand/or roughness), type and weight of cargo, and/or other parameters. Byway of a non-limiting illustration, responsive to carrying delicateobjects, speed may be reduced and/or movement control precision may beincreased. The user interface controls may be adapted to provideincreased precision control functionality. Increasing precision controlfunctionality may include providing coarse/fine movement controls.Responsive to operating in the presence of external forces (e.g., wind,currents, and/or slope), the control configured to control movement inthe direction of the external force (e.g., along river) may be adaptedto provide extra acceleration when applied in a direction against theforce (e.g., against current) and/or reduced acceleration when appliedin a direction coinciding with the external force (e.g., with thecurrent). The configuration and/or presentation of the controls (e.g.,their orientation and/or size) may be altered by the presence of theseand/or other external forces.

In some implementations, the modification detection information may beprovided by a user (e.g., via changes to a configuration register and/ora command) and/or detected automatically based, for example, on anoutput of robot's orientation sensor.

At operation 828, interface of the robotic controller may be adjustedconsistent with the modified robot configuration as described, forexample, with respect to operation 802 of FIG. 8A above.

At operation 830, the robot may be operated using the adjusted interfaceconfiguration determined at operation 828. In some implementations,operations 830 may comprise controlling speed and direction of the rover220 of FIG. 2C using sliders 302, 304 and/or the slider 302 and the knob324 of FIG. 3 and/or controlling the robotic apparatus 400 of FIG. 4Ausing control elements 452, 458, 462, 464 shown in FIG. 4B and/orcontrol elements 478, 482 shown in FIG. 4C.

FIG. 9 is a logical flow diagram illustrating a method of training arobotic apparatus using an adaptive remoter controller apparatus, inaccordance with one or more implementations. Operations of method 900may be utilized in training a robot to perform an action such as, forexample, following a target trajectory.

At operation 902, a data communication may be established with a robot.In some implementations, the communication may comprise communicationbetween the robot and a robot remote controller (e.g., 604 in FIG. 6)via wired and/or wireless link.

At operation 904, a robot configuration may be determined. In someimplementations, determination of the robot's configuration may compriseoperation 702 of method 700 of FIG. 7 described above.

At operation 906, an interface of a robot controller may be configuredto conform to the robot configuration. In some implementations, therobot configuration may be characterized by a number of joints, a numberof motor actuators, orientation of joints and/or motor actuators, and/orother information associated with configuration. Arrangement of theremote control interface may comprise disposing control element (e.g.,sliders 302, 304 in FIG. 3) parallel to the respective motion components(e.g., 232, 234 of FIG. 2C).

At operation 908, training may commence. A training goal may comprisedirecting the robot to follow a target trajectory.

During training, at operation 910, intuitive correspondence may bedeveloped between the control commands and the resultant action by therobot. The development of the intuitive correspondence may befacilitated based on the conforming configuration of the controllerinterface obtained at operations 906. By way of non-limitingillustration, using controller interface wherein motion of controlelements (e.g., the sliders 302, 304 in FIG. 3) matches orientation ofrobot movements (e.g., motion components denoted by arrows 232, 234 ofFIG. 2C) may enable user to provide more timely training input, reducenumber of erroneous commands due to, e.g., user confusion.

At operation 912, the training goal may be attained. In someimplementations, the goal attainment may be determined based on therobot navigating the target directory with target performance. In one ormore implementations, training performance may be determined based on adiscrepancy measure between the actual robot trajectory and the targettrajectory. The discrepancy measure may comprise one or more of maximumdeviation, maximum absolute deviation, average absolute deviation, meanabsolute deviation, mean difference, root mean squared error, cumulativedeviation, and/or other measures.

One or more of the methodologies comprising adaptation of remote controluser interface described herein may facilitate training and/or operationof robotic devices. In some implementations, a user interface configuredto match configuration of the robot may enable users to provide moretimely training input, reduce number of erroneous commands due to, e.g.,user confusion. Such development of intuitive correspondence between thecontroller interface and the robot behaved (e.g., movements)improvements may reduce training time and/or improve training accuracy.In some applications, adaptively configured user interface may freeusers from the need to re-program remote control devices for everyindividual robot configuration thereby enabling a wide population ofusers without specialized robotic programming skills to train andoperate a wider variety of robots.

In some implementations, remote interface adaptation due to detectedrobot component failures may improve user experience while operaterobotic devices by, for example, disabling controls for failedcomponents.

It will be recognized that while certain aspects of the disclosure aredescribed in terms of a specific sequence of steps of a method, thesedescriptions are only illustrative of the broader methods of thedisclosure, and may be modified as required by the particularapplication. Certain steps may be rendered unnecessary or optional undercertain circumstances. Additionally, certain steps or functionality maybe added to the disclosed implementations, or the order of performanceof two or more steps permuted. All such variations are considered to beencompassed within the disclosure disclosed and claimed herein.

While the above detailed description has shown, described, and pointedout novel features of the disclosure as applied to variousimplementations, it will be understood that various omissions,substitutions, and changes in the form and details of the device orprocess illustrated may be made by those skilled in the art withoutdeparting from the disclosure. This description is in no way meant to belimiting, but rather should be taken as illustrative of the generalprinciples of the technology. The scope of the disclosure should bedetermined with reference to the claims.

What is claimed is:
 1. A non-transitory machine-readable storage mediumhaving instructions embodied thereon, the instructions being executableto perform a method for controlling a robotic apparatus, the methodcomprising: establishing a data connection to the robotic apparatus;receiving information related to a phenotype of the robotic apparatus;and issuing a command to a user interface apparatus, the user interfaceapparatus executing an action based on the command, the commandindicative of at least one configuration associated with theinformation; wherein: the user interface apparatus comprises a displayapparatus comprising at least one control configured to relay user inputto the robotic apparatus; executing the action causes the user interfaceapparatus to alter a representation of the at least one controlconsistent with the information; the phenotype is characterized by oneor both of (i) a hardware configuration of the robotic apparatus or (ii)an operational configuration of the robotic apparatus; the informationis based on a statistical parameter related to a plurality of actionsexecuted by the robotic apparatus responsive to a plurality of usercommands relayed by the control; and individual ones of the plurality ofactions are determined based on one or both of the hardwareconfiguration of the robotic apparatus or the operational configurationof the robotic apparatus.
 2. The non-transitory machine-readable storagemedium of claim 1, wherein the command is configured to be issuedautomatically absent an explicit request by the user.
 3. Thenon-transitory machine-readable storage medium of claim 1, wherein: theat least one control comprises: (1) a first slider configured to relateforward and reverse motion commands; (2) a second slider configured torelate left and right turn commands; and (3) reverse motion commands;and main axes of the first slider and the second slider are disposedperpendicular with one another.
 4. The non-transitory machine-readablestorage medium of claim 1, wherein: the phenotype is characterized byone or both of (i) a hardware configuration of the robotic device or(ii) operational configuration of the robotic device; and theinformation is configured to relate modification of one or both of thehardware configuration or the operational configuration of the roboticdevice.
 5. The non-transitory machine-readable storage medium of claim4, wherein the hardware configuration of the robotic device comprisesone or more of a number of motor actuators, a rotation axis orientationfor individual actuators, or a number of actuators configured to beactivated simultaneously.
 6. The non-transitory machine-readable storagemedium of claim 4, wherein the operational configuration of the roboticdevice comprises one or more of a number of motor actuators, a rotationaxis orientation for individual actuators, or a number of actuatorsconfigured to be activated simultaneously.
 7. The non-transitorymachine-readable storage medium of claim 1, wherein: the phenotype ischaracterized by a hardware configuration of the robotic apparatus; theinformation is based on a statistical parameter related to a pluralityof actions executed by the robotic apparatus responsive to a pluralityof user commands relayed by the control; and individual ones of theplurality of actions configured based on at least one of the hardwareconfiguration of the robotic apparatus.
 8. The non-transitorymachine-readable storage medium of claim 7, wherein altering of therepresentation of the at least one control consistent with theinformation comprises rendering the control on the display apparatusconsistent with the hardware configuration.
 9. The non-transitorymachine-readable storage medium of claim 8, wherein: the roboticapparatus comprises one or both of a wheel or a joint, characterized byaxis of rotation; the hardware configuration is configured to conveyplacement of the axis with respect to a reference direction; and thealtering of the representation of the at least one control consistentwith the information comprises disposing the control on the displayapparatus at an orientation matching the placement of the axis withrespect to the reference direction.
 10. The non-transitorymachine-readable storage medium of claim 9, wherein the referencedirection comprises orientation of one of display dimensions.
 11. Thenon-transitory machine-readable storage medium of claim 8, wherein theuser interface apparatus comprises one or more of touch-sensinginterface, a contactless motion sensing interface, or a radio frequencywireless interface.
 12. The non-transitory machine-readable storagemedium of claim 1, wherein: the robotic apparatus comprises a humanoidrobot comprising (1) a first joint configured to be rotated with respectto a first axis and (2) a second joint configured to be rotated withrespect to a second axis, a non-zero angle existing between the firstaxis and the second axis; and the altering of the representation of theat least one control is configured to dispose a first control elementand a second control element adapted to control the first joint and thesecond joint, respectively, at the non-zero angle.
 13. A non-transitorymachine-readable storage medium having instructions embodied thereon,the instructions being executable to perform a method for controlling arobotic device, the method comprising: establishing a data connection tothe robotic device; receiving information related to a phenotype of therobotic device; and issuing a command to a user interface apparatus, theuser interface apparatus executing an action based on the command, thecommand indicative of at least one configuration associated with theinformation; wherein: the user interface apparatus comprises a displayapparatus comprising at least one control configured to relay user inputto the robotic device; executing the action causes the user interfaceapparatus to alter a representation of the at least one controlconsistent with the information; the robotic device comprises at leastone actuator characterized by an axis of motion; and the information isconfigured to relate an orientation of the axis of motion with respectto a reference orientation.
 14. The non-transitory machine-readablestorage medium of claim 13, wherein: the reference orientation comprisesa geographical coordinate; and the information comprises a computerdesign file of the robotic device, the computer design file comprising adescription of the actuator and the axis of motion.
 15. Thenon-transitory machine-readable storage medium of claim 13, wherein: thereference orientation comprises an axis of the robotic device; thedisplay apparatus is characterized by a default orientation; andaltering the representation of the at least one control consistent withthe information comprises: determining an angle between the referenceorientation and the axis of motion; and positioning the at least onecontrol on the display apparatus at the angle relative the defaultorientation.
 16. The non-transitory machine-readable storage medium ofclaim 15, wherein: the phenotype is characterized by a hardwareconfiguration of the robotic device, the hardware configurationcomprising information related to one or more of the actuator and theaxis of the robotic device; the information is based on a statisticalparameter related to a plurality of actions executed by the roboticdevice responsive to a plurality of user commands relayed by thecontrol; and individual ones of the plurality of actions configuredbased on at least one of the hardware configuration of the roboticdevice.
 17. The non-transitory machine-readable storage medium of claim16, wherein the command is configured to be issued automatically absentan explicit request by the user.
 18. The non-transitory machine-readablestorage medium of claim 17, wherein altering of the representation ofthe at least one control consistent with the information comprisesrendering the control on the display apparatus consistent with thehardware configuration.
 19. The non-transitory machine-readable storagemedium of claim 13, wherein the reference direction comprisesorientation of one of display dimensions.
 20. A non-transitorymachine-readable storage medium having instructions embodied thereon,the instructions being executable to perform a method for controlling arobotic device, the method comprising: establishing a data connection tothe robotic device; receiving information related to a phenotype of therobotic device; and issuing a command to a user interface apparatus, theuser interface apparatus executing an action based on the command, thecommand indicative of at least one configuration associated with theinformation; wherein: the user interface apparatus comprises a displayapparatus comprising at least one control configured to relay user inputto the robotic device; executing the action causes the user interfaceapparatus to alter a representation of the at least one controlconsistent with the information; the robotic device comprises first andsecond actuators configured to displace at least a portion of therobotic device in a first direction and a second direction,respectively; the information comprises parameters of the firstdirection and the second direction; the at least one control comprises afirst translational motion control and a second translational motioncontrol associated with the first actuator and the second actuator,respectively; and the act of altering the representation of the at leastone control consistent with the information comprises: positioning thefirst translational motion control at the first direction; andpositioning the second translational motion control at the seconddirection, the second direction being perpendicular to the firstdirection.
 21. The non-transitory machine-readable storage medium ofclaim 20, wherein: the robotic device is characterized by a defaultorientation; and the first direction and the second direction comprise adirection of longitudinal and transverse motions relative to the defaultorientation.
 22. An apparatus configured to control a robotic device,the apparatus comprising: a non-transitory machine-readable storagemedium having machine-readable instructions embodied thereon; and one ormore physical processors configured by the machine-readable instructionsto: establish a data connection to the robotic device; receiveinformation related to a phenotype of the robotic device; and issue acommand to a user interface apparatus, the user interface apparatusexecuting an action based on the command, the command indicative of atleast one configuration associated with the information; wherein: theuser interface apparatus comprises a display apparatus comprising atleast one control configured to relay user input to the robotic device;executing the action causes the user interface apparatus to alter arepresentation of the at least one control consistent with theinformation; the robotic device is characterized by a defaultorientation; the robotic device comprises first actuator configured torotate at least a portion of the robotic device about a first axisconfigured vertically with respect to the default orientation; thesecond actuator is configured to move the robotic device in alongitudinal direction relative the default orientation; and the act ofaltering the representation of the at least one control consistent withthe information comprises: providing a rotational control elementconfigured to receive an indication related to rotation of the least aportion of the robotic device about the first axis disposed verticallywith respect to the default orientation, an axis of the rotationalcontrol element being disposed parallel to the first axis; and providinga longitudinal control element configured to receive an indicationrelated to displacement of the robotic device along the longitudinaldirection, an axis of the longitudinal control element being disposedparallel to the longitudinal direction.
 23. The apparatus of claim 22,wherein the rotational control element comprises a knob having an axisof rotation configured such that there is a match between (1) a mutualorientation of the axis of rotation and the longitudinal direction and(2) a mutual orientation of the axis of rotation and the defaultorientation of the robotic device.
 24. The apparatus of claim 22,wherein: the phenotype is characterized by one or both of (i) a hardwareconfiguration of the robotic device—or (ii) an operational configurationof the robotic device; the information is based on a statisticalparameter related to a plurality of actions executed by the roboticdevice responsive to a plurality of user commands relayed by thecontrol; and altering of the representation of the at least one controlconsistent with the information comprises rendering the control on thedisplay apparatus consistent with the hardware configuration.
 25. Theapparatus of claim 24, wherein: the robotic device comprises one or bothof a wheel or a joint, characterized by axis of rotation; the hardwareconfiguration is configured to convey placement of the axis of rotationwith respect to a reference direction; and the altering of therepresentation of the at least one control consistent with theinformation comprises disposing the control on the display apparatus atan orientation matching the placement of the axis of rotation withrespect to the reference direction.
 26. The apparatus of claim 24,wherein the user interface apparatus comprises one or more oftouch-sensing interface, a contactless motion sensing interface, or aradio frequency wireless interface.